Papers with victim model
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor attacks insert additional contents into normal samples as triggers, causing detection and blocking of backdoors. |
| Approach: | They propose to use syntactic structure as trigger in textual backdoor attacks . they propose to achieve similar attack performance but have higher invisibility . |
| Outcome: | The proposed method achieves almost 100% success rate but has higher invisibility and stronger resistance to defenses than the insertion-based methods. |
An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to generating NMT adversarial examples inject perturbations into source sentences or target translations to improve the robustness of NMT models. |
| Approach: | They investigate adversarial attack on NMT from two aspects: languages (the source vs. the target language) and positions (front v. rear). |
| Outcome: | The proposed approach is more effective than adversarial attacks by sampling positions randomly or according to gradients. |
SHARP: Search-Based Adversarial Attack for Structured Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | SHARP is a new attack method for structured prediction models that solves several challenges. |
| Approach: | They propose a black-box adversarial attack method that uses a search-based optimization problem to attack adversarials. |
| Outcome: | The proposed method performs more potent attack than pioneer arts on two structured prediction tasks. |
“Yes, My LoRD.” Guiding Language Model Extraction with Locality Reinforced Distillation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for model extraction attacks on large language models are inadequate . existing methods neglect the inconsistency between training tasks and LLM alignment . |
| Approach: | They propose a model extraction algorithm that uses a policy-gradient-style training task to guide the crafting of preference for the local model. |
| Outcome: | The proposed algorithm reduces query complexity while mitigating watermark protection . it can extract various state-of-the-art commercial LLMs while minimizing query complexity . |
Attack Named Entity Recognition by Entity Boundary Interference (2024.lrec-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a cornerstone natural language processing task . despite its robustness, studies on its robustity are lacking. |
| Approach: | They propose a one-word modification NER attack that strategically inserts a new boundary into the sentence and triggers the model to make a wrong recognition. |
| Outcome: | The proposed method is effective on English and Chinese models with 70%-90% success rate. |
RMLM: A Flexible Defense Framework for Proactively Mitigating Word-level Adversarial Attacks (2023.acl-long)
Copied to clipboard
| Challenge: | Existing defenses focus on improving robustness of the victim model in training, but neglect to mitigate adversarial attacks during inference. |
| Approach: | They propose a framework that confuses attackers and corrects adversarial contexts . their framework helps improve the robustness of the victim model during inference . |
| Outcome: | The proposed framework improves the robustness of the victim model in training . it also corrects abnormal contexts in the representation level and filtering out examples . |
Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are used for natural language processing (NLP) but when they are deployed as a service, they can suffer from different attacks . |
| Approach: | They propose two defence strategies to protect the target model from adversarial attacks . they show that model extraction and adversarially transferable attacks can be effective . |
| Outcome: | The extracted model can lead to highly transferable adversarial attacks against the target model. |
Adversarial Attack and Defense of Structured Prediction Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building effective adversarial attackers focus on classification problems. |
| Approach: | They propose a framework that learns to attack a structured prediction model with feedbacks from multiple reference models. |
| Outcome: | The proposed framework is able to attack state-of-the-art models and boost them with training . it is based on a sequence-to-sequence model with feedbacks from multiple reference models . |
DITTO: A Spoofing Attack Framework on Watermarked LLMs via Knowledge Distillation (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate coherent, human-like text at scale, but raises concerns about authenticity and trust. |
| Approach: | They propose a threat of watermark spoofing that allows a malicious model to generate text containing the authentic-looking watermark of a trusted, victim model. |
| Outcome: | The proposed attack repurposes watermark radioactivity from a discoverable trait into an attack vector and replicates it. |
Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text embedding models have significantly streamlined the process of generating embeddables. |
| Approach: | They develop a transfer attack method that uses a surrogate model to mimic the victim model's behavior and infers sensitive information from embeddings without direct access. |
| Outcome: | The proposed method outperforms existing methods and reveals potential privacy vulnerabilities in embedding technologies. |
CT-GAT: Cross-Task Generative Adversarial Attack based on Transferability (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural network models are vulnerable to adversarial examples, and current methods based on adversarially transferable models rely on substitute models, which can be impractical and costly in real-world scenarios due to the unavailability of training data and the victim model’s structural details. |
| Approach: | They propose a novel approach that directly constructs adversarial examples by extracting transferable features across various tasks. |
| Outcome: | The proposed approach achieves superior attack performance with small cost on ten datasets and demonstrates that it is a novel approach. |
Mitigating Data Poisoning in Text Classification with Differential Privacy (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Data poisoning attacks can plant a backdoor in a model by injecting poisoned examples into training data, causing the model to misclassify test instances which include a specific pattern. |
| Approach: | They propose a generic defence mechanism that makes training robust to poisoning attacks by smoothing the gradient from each training example. |
| Outcome: | The proposed method is highly effective in mitigating, or even eliminating, poisoning attacks on text classification, with only a small cost in predictive accuracy. |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
A Middle Path for On-Premises LLM Deployment: Preserving Privacy Without Sacrificing Model Confidentiality (2025.emnlp-main)
Copied to clipboard
| Challenge: | Privacy-sensitive users require deploying large language models within their own infrastructure (on-premises) vulnerabilities in local environments can lead to unauthorized access and potential model theft. |
| Approach: | They propose a framework that secures a few bottom layers in a secure environment . they propose metric to optimize trade-off between protection and customization flexibility . |
| Outcome: | The proposed framework outperforms baselines on five models with 1.3B to 70B parameters. |
Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are vulnerable to backdoors that use long-form reasoning to generate a specific word, choice, or class. |
| Approach: | They propose a mechanism that allows LLMs to develop critical thinking behaviors and detect backdoors by a two-stage fine-tuning. |
| Outcome: | The proposed mechanism exhibits strong cross-domain and cross-task generalization. |
The Thieves on Sesame Street are Polyglots - Extracting Multilingual Models from Monolingual APIs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated that deployed NLP models can be stolen by adversaries by querying victim models with gibberish input data that consists of random sequences of words. |
| Approach: | They propose to extract a local copy of a monolingual victim model from an API and query it with gibberish input data paired with the victim's labels. |
| Outcome: | The extracted model learns the task from the monolingual victim, but it generalizes far better than the victim to several other languages. |
Adversarial Reprogramming of Text Classification Neural Networks (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can cause a machine learning model to misclassify a sample from the classifier's input domain. |
| Approach: | They propose a context-based vocabulary remapping method that performs a computationally inexpensive input transformation to reprogram a victim classification model for a new set of sequences. |
| Outcome: | The proposed method performs a cost-effective input transformation to reprogram a model for a new set of sequences without altering the network architecture or parameters. |
Query-Efficient Black-Box Red Teaming via Bayesian Optimization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating test cases and querying fail to be query-efficient . generative models can be used for open-domain dialogue, prompt continuation, text-to-image generation . |
| Approach: | They propose a query-efficient method that iteratively finds diverse positive test cases leading to model failures by utilizing user input and past evaluations. |
| Outcome: | The proposed method finds a significantly larger number of diverse positive test cases under limited query budget than baseline methods. |
Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial text attacks rely on abundant access to shared internal features and numerous queries, limited to a single task type. |
| Approach: | They propose a black-box attack that exploits the transferability of adversarial texts . they use a deep-level substitute model trained in a plug-and-play manner for text classification . |
| Outcome: | The proposed attack can target multiple tasks with minimal perturbations . it can target commercial APIs, large language models, and image-generation models . |
BITE: Textual Backdoor Attacks with Iterative Trigger Injection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to defend against backdoor attacks are based on model stealing, model thieving and training data extraction attacks. |
| Approach: | They propose a backdoor attack that poisons training data to establish strong correlations between the target label and a set of “trigger words” These trigger words are iteratively identified and injected into the target-label instances through natural word-level perturbations. |
| Outcome: | The proposed attack is significantly more effective than baseline methods while maintaining decent stealthiness, raising alarm on the usage of untrusted training data. |
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data . |
| Approach: | They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data. |
| Outcome: | The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. |
MeaeQ: Mount Model Extraction Attacks with Efficient Queries (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on limited-query budget settings and adopt random sampling or active learning-based sampling strategies on publicly available, unannotated data sources. |
| Approach: | They propose a model extraction attack with efficient Queries that uses a zero-shot sequence inference classifier to filter task-relevant data from a public text corpus instead of a problem domain-specific dataset. |
| Outcome: | The proposed method achieves higher similarity to the victim model than baselines while requiring fewer queries. |
Fooling the Textual Fooler via Randomizing Latent Representations (2024.findings-acl)
Copied to clipboard
| Challenge: | Several adversarial attacks can compromise the model without accessing the model architecture or model parameters (i.e., a blackbox setting) Several studies have revealed that deep NLP models are vulnerable to adversarials that slightly perturb the input to cause the models to misbehave. |
| Approach: | They propose a lightweight and attack-agnostic defense that perplexes the process of generating an adversarial example in query-based black-box attacks. |
| Outcome: | The proposed defense is lightweight and attack-agnostic and does not necessitate additional computational overhead during training nor does it rely on assumptions about the potential adversarial perturbation set while having a negligible impact on the model’s accuracy. |
Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Model NLP models are often trained on datasets from untrusted platforms, posing significant risks of data poisoning attacks. |
| Approach: | They propose a retraining-free method that selectively replaces modules in the victim model based on a trade-off signal between utility and backdoor. |
| Outcome: | The proposed method outperforms even the strongest defense baseline against challenging attacks like LWS. |